<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.4: http://docutils.sourceforge.net/" />
<title>Build slave documentation</title>
<meta name="author" content="Steven Mohr" />
<meta name="date" content="$Date$" />
<style type="text/css">

/*
:Author: David Goodger
:Contact: goodger@users.sourceforge.net
:Date: $Date: 2005-12-18 01:56:14 +0100 (Sun, 18 Dec 2005) $
:Revision: $Revision: 4224 $
:Copyright: This stylesheet has been placed in the public domain.

Default cascading style sheet for the HTML output of Docutils.

See http://docutils.sf.net/docs/howto/html-stylesheets.html for how to
customize this style sheet.
*/

/* used to remove borders from tables and images */
.borderless, table.borderless td, table.borderless th {
  border: 0 }

table.borderless td, table.borderless th {
  /* Override padding for "table.docutils td" with "! important".
     The right padding separates the table cells. */
  padding: 0 0.5em 0 0 ! important }

.first {
  /* Override more specific margin styles with "! important". */
  margin-top: 0 ! important }

.last, .with-subtitle {
  margin-bottom: 0 ! important }

.hidden {
  display: none }

a.toc-backref {
  text-decoration: none ;
  color: black }

blockquote.epigraph {
  margin: 2em 5em ; }

dl.docutils dd {
  margin-bottom: 0.5em }

/* Uncomment (and remove this text!) to get bold-faced definition list terms
dl.docutils dt {
  font-weight: bold }
*/

div.abstract {
  margin: 2em 5em }

div.abstract p.topic-title {
  font-weight: bold ;
  text-align: center }

div.admonition, div.attention, div.caution, div.danger, div.error,
div.hint, div.important, div.note, div.tip, div.warning {
  margin: 2em ;
  border: medium outset ;
  padding: 1em }

div.admonition p.admonition-title, div.hint p.admonition-title,
div.important p.admonition-title, div.note p.admonition-title,
div.tip p.admonition-title {
  font-weight: bold ;
  font-family: sans-serif }

div.attention p.admonition-title, div.caution p.admonition-title,
div.danger p.admonition-title, div.error p.admonition-title,
div.warning p.admonition-title {
  color: red ;
  font-weight: bold ;
  font-family: sans-serif }

/* Uncomment (and remove this text!) to get reduced vertical space in
   compound paragraphs.
div.compound .compound-first, div.compound .compound-middle {
  margin-bottom: 0.5em }

div.compound .compound-last, div.compound .compound-middle {
  margin-top: 0.5em }
*/

div.dedication {
  margin: 2em 5em ;
  text-align: center ;
  font-style: italic }

div.dedication p.topic-title {
  font-weight: bold ;
  font-style: normal }

div.figure {
  margin-left: 2em ;
  margin-right: 2em }

div.footer, div.header {
  clear: both;
  font-size: smaller }

div.line-block {
  display: block ;
  margin-top: 1em ;
  margin-bottom: 1em }

div.line-block div.line-block {
  margin-top: 0 ;
  margin-bottom: 0 ;
  margin-left: 1.5em }

div.sidebar {
  margin-left: 1em ;
  border: medium outset ;
  padding: 1em ;
  background-color: #ffffee ;
  width: 40% ;
  float: right ;
  clear: right }

div.sidebar p.rubric {
  font-family: sans-serif ;
  font-size: medium }

div.system-messages {
  margin: 5em }

div.system-messages h1 {
  color: red }

div.system-message {
  border: medium outset ;
  padding: 1em }

div.system-message p.system-message-title {
  color: red ;
  font-weight: bold }

div.topic {
  margin: 2em }

h1.section-subtitle, h2.section-subtitle, h3.section-subtitle,
h4.section-subtitle, h5.section-subtitle, h6.section-subtitle {
  margin-top: 0.4em }

h1.title {
  text-align: center }

h2.subtitle {
  text-align: center }

hr.docutils {
  width: 75% }

img.align-left {
  clear: left }

img.align-right {
  clear: right }

ol.simple, ul.simple {
  margin-bottom: 1em }

ol.arabic {
  list-style: decimal }

ol.loweralpha {
  list-style: lower-alpha }

ol.upperalpha {
  list-style: upper-alpha }

ol.lowerroman {
  list-style: lower-roman }

ol.upperroman {
  list-style: upper-roman }

p.attribution {
  text-align: right ;
  margin-left: 50% }

p.caption {
  font-style: italic }

p.credits {
  font-style: italic ;
  font-size: smaller }

p.label {
  white-space: nowrap }

p.rubric {
  font-weight: bold ;
  font-size: larger ;
  color: maroon ;
  text-align: center }

p.sidebar-title {
  font-family: sans-serif ;
  font-weight: bold ;
  font-size: larger }

p.sidebar-subtitle {
  font-family: sans-serif ;
  font-weight: bold }

p.topic-title {
  font-weight: bold }

pre.address {
  margin-bottom: 0 ;
  margin-top: 0 ;
  font-family: serif ;
  font-size: 100% }

pre.literal-block, pre.doctest-block {
  margin-left: 2em ;
  margin-right: 2em ;
  background-color: #eeeeee }

span.classifier {
  font-family: sans-serif ;
  font-style: oblique }

span.classifier-delimiter {
  font-family: sans-serif ;
  font-weight: bold }

span.interpreted {
  font-family: sans-serif }

span.option {
  white-space: nowrap }

span.pre {
  white-space: pre }

span.problematic {
  color: red }

span.section-subtitle {
  /* font-size relative to parent (h1..h6 element) */
  font-size: 80% }

table.citation {
  border-left: solid 1px gray;
  margin-left: 1px }

table.docinfo {
  margin: 2em 4em }

table.docutils {
  margin-top: 0.5em ;
  margin-bottom: 0.5em }

table.footnote {
  border-left: solid 1px black;
  margin-left: 1px }

table.docutils td, table.docutils th,
table.docinfo td, table.docinfo th {
  padding-left: 0.5em ;
  padding-right: 0.5em ;
  vertical-align: top }

table.docutils th.field-name, table.docinfo th.docinfo-name {
  font-weight: bold ;
  text-align: left ;
  white-space: nowrap ;
  padding-left: 0 }

h1 tt.docutils, h2 tt.docutils, h3 tt.docutils,
h4 tt.docutils, h5 tt.docutils, h6 tt.docutils {
  font-size: 100% }

tt.docutils {
  background-color: #eeeeee }

ul.auto-toc {
  list-style-type: none }

</style>
</head>
<body>
<div class="document" id="build-slave-documentation">
<h1 class="title">Build slave documentation</h1>
<table class="docinfo" frame="void" rules="none">
<col class="docinfo-name" />
<col class="docinfo-content" />
<tbody valign="top">
<tr><th class="docinfo-name">Author:</th>
<td>Steven Mohr</td></tr>
<tr><th class="docinfo-name">Date:</th>
<td>$Date$</td></tr>
<tr><th class="docinfo-name">Revision:</th>
<td>$Rev$</td></tr>
<tr><th class="docinfo-name">Status:</th>
<td>Draft</td></tr>
</tbody>
</table>
<div class="contents topic">
<p class="topic-title first"><a id="contents" name="contents">Contents</a></p>
<ul class="simple">
<li><a class="reference" href="#structure-of-a-build-slave-directory" id="id1" name="id1">Structure of a build slave directory</a></li>
<li><a class="reference" href="#how-to-create-an-unix-slave" id="id2" name="id2">How-To create an Unix slave</a><ul>
<li><a class="reference" href="#example-for-debian-i686" id="id3" name="id3">Example for Debian i686</a></li>
<li><a class="reference" href="#ubuntu" id="id4" name="id4">Ubuntu</a></li>
<li><a class="reference" href="#solaris" id="id5" name="id5">Solaris</a><ul>
<li><a class="reference" href="#solaris-py23" id="id6" name="id6">Solaris Py23</a></li>
</ul>
</li>
</ul>
</li>
<li><a class="reference" href="#how-to-create-a-windows-slave" id="id7" name="id7">How-To create a windows slave</a></li>
<li><a class="reference" href="#known-problems" id="id8" name="id8">Known problems</a></li>
</ul>
</div>
<div class="section">
<h1><a class="toc-backref" href="#id1" id="structure-of-a-build-slave-directory" name="structure-of-a-build-slave-directory">Structure of a build slave directory</a></h1>
<p>This is the structure of the directory:</p>
<img alt="TG-Buildslave_dirstruct.png" src="TG-Buildslave_dirstruct.png" style="width: 233.0px; height: 259.0px;" />
</div>
<div class="section">
<h1><a class="toc-backref" href="#id2" id="how-to-create-an-unix-slave" name="how-to-create-an-unix-slave">How-To create an Unix slave</a></h1>
<div class="section">
<h2><a class="toc-backref" href="#id3" id="example-for-debian-i686" name="example-for-debian-i686">Example for Debian i686</a></h2>
<p>I've used a netinst image as the base of my buildslave but you can use every image.
For the next steps you need to be root:</p>
<ol class="arabic simple">
<li>open the /etc/apt/sources.list and remove your cdrom drive (else apt-get asks you everytime to insert the install cd).</li>
<li>install the following packages via apt-get install</li>
</ol>
<blockquote>
<ul class="simple">
<li>zlib1g-dev</li>
<li>build-essential</li>
<li>python-all-dev</li>
<li>python-sqlite2</li>
<li>python-twisted</li>
<li>python-turbogears</li>
<li>subversion</li>
<li>mercurial</li>
<li>libbz2-dev</li>
<li>bzip2</li>
</ul>
</blockquote>
<ol class="arabic simple" start="3">
<li>install virtualenv using easy_install-2.4 virtualenv</li>
<li>rename /usr/bin/virtualenv to /usr/bin/virtualenv-2.4 (the standard python version of my image is 2.4 so I rename it to virtualenv-2.4)</li>
<li>install python2.5 and python2.5-dev via apt-get</li>
<li>download setuptools install script via wget <a class="reference" href="http://peak.telecommunity.com/dist/ez_setup.py">http://peak.telecommunity.com/dist/ez_setup.py</a></li>
<li>run python2.5 ez_setup.py</li>
<li>easy_install-2.5 virtualenv</li>
<li>rename the new virtualenv script (mv /usr/bin/virtualenv /usr/bin/virtualenv-2.5)</li>
</ol>
<p>I recommend to install buildbot into an own virtual environment. This has a a lot of advantages but the biggest is that you can switch back to your normal user so
for the next steps you don't need to be root</p>
<ol class="arabic simple" start="10">
<li>virtualenv-2.5 builbot</li>
<li>activate our new virtual enviornment: source buildbot/bin/activate</li>
<li>easy_install buildbot</li>
<li>easy_install twisted</li>
</ol>
<p>Now we are able to create our first build slave</p>
<ol class="arabic simple" start="14">
<li>buildbot create-slave slave25 &lt;master&gt; &lt;name&gt; &lt;passwd&gt;</li>
</ol>
<p>&lt;master&gt; is IPOfTheBuildMaster:Port. &lt;name&gt; is the slave and should fit this pattern: &lt;OS+architecture&gt;_py&lt;python version&gt;, f.e. Ubuntu64_py25.
&lt;passwd&gt; is the password to access the build slaves.</p>
<ol class="arabic simple" start="15">
<li>change directory to slave25 and create a symlink to virtualenv (cd slave25 ln -s /usr/bin/virtualenv-2.5 virtualenv)</li>
</ol>
<p>We almost have our first buildslave there's only one thing we have to do before we're finished: Because of an limitation in the mercurial class of BuildBot we have
to create 2 directories.</p>
<ol class="arabic simple" start="16">
<li>mkdir TG2full, mkdir TG2full/build, mkdir TG2full/build/webhelpers TG2full/build/pylons</li>
</ol>
<p>Our first slave is finished. The second is as easy as the first, so let's start:</p>
<ol class="arabic simple" start="17">
<li>buildbot create-slave slave24 &lt;master&gt; &lt;name&gt; &lt;passwd&gt;</li>
</ol>
<p>&lt;master&gt; is the same as before. You only have to change the version number in &lt;name&gt; and define a new password.</p>
<ol class="arabic simple" start="18">
<li>change directory to slave24 and create a symlink to virtualenv (cd slave24 ln -s /usr/bin/virtualenv-2.4 virtualenv)</li>
<li>mkdir TG2full, mkdir TG2full/build, mkdir TG2full/build/webhelpers TG2full/build/pylons</li>
<li>switch back to the root directory of our slaves</li>
</ol>
<p>The creation of our last slave isn't that easy. The first problem is that there aren't any Py2.3 packages in the repo, so you have to build it from source.</p>
<ol class="arabic simple" start="21">
<li>wget <a class="reference" href="http://www.python.org/ftp/python/2.3.7/Python-2.3.7.tgz">http://www.python.org/ftp/python/2.3.7/Python-2.3.7.tgz</a></li>
<li>untar -xf Python-2.3.7.tgz</li>
<li>cd Python-2.3.7</li>
<li>./configure</li>
</ol>
<p>For the next steps you need to be root:</p>
<ol class="arabic simple" start="25">
<li>make</li>
<li>make install</li>
<li>python23 ez_setup.py</li>
<li>easy_install-2.3 virtualenv==0.9</li>
<li>rename the new virtualenv script (mv /usr/local/bin/virtualenv /usr/local/bin/virtualenv-2.3)</li>
<li>remove /usr/local/bin from $PATH</li>
<li>buildbot create-slave slave23 ...</li>
<li>change directory to slave23 and create a symlink to virtualenv (cd slave23 ln -s /usr/local/bin/virtualenv-2.3 virtualenv)</li>
</ol>
<p>So that's almost everything; one last step</p>
<ol class="arabic simple" start="33">
<li>download tg_sbuildsteps.py from svn (<a class="reference" href="http://svn.turbogears.org/build/buildmaster/tg_sbuildsteps.py">http://svn.turbogears.org/build/buildmaster/tg_sbuildsteps.py</a>) and copy it to
(directory of your python installation)/lib/python25/site-packages/buildbot-0.7.7-py2.5.egg/buildbot/slaves.
Rename commands.py to _commands.py and run cat _commands.py tg_mbuildsteps.py &gt; commands.py</li>
</ol>
<p>That's it, you have your first three build slave. Now you can start them with buildbot start &lt;DirOfSlave&gt;.</p>
</div>
<div class="section">
<h2><a class="toc-backref" href="#id4" id="ubuntu" name="ubuntu">Ubuntu</a></h2>
<p>Under Ubuntu it's the same the only difference is that you have to use sudo for all root commands.</p>
</div>
<div class="section">
<h2><a class="toc-backref" href="#id5" id="solaris" name="solaris">Solaris</a></h2>
<p>I've used OpenSolaris to create the image. It's important to use the Developer Edition. This includes a few packages which you really need.</p>
<p>The first step is to download and install all packages you need for Python, Subversion and Mercurial. It's:</p>
<blockquote>
<ul class="simple">
<li>apache2</li>
<li>bzip2</li>
<li>db-4.2</li>
<li>expat</li>
<li>gcc-3.4.6</li>
<li>gdbm</li>
<li>libiconv</li>
<li>libintl</li>
<li>libxml</li>
<li>mercurial</li>
<li>ncurses</li>
<li>neon</li>
<li>openssl</li>
<li>python-2.5.1</li>
<li>readline</li>
<li>sqlite</li>
<li>subversion</li>
<li>swig</li>
<li>tcl</li>
<li>tk</li>
<li>wget</li>
<li>zlib</li>
</ul>
</blockquote>
<p>This packages are at sunfreeware.com.</p>
<p>The next steps are similar to Debian. You have to install virtualenv, rename it, install setuptools for Py2.5, install virtualenv,
rename it, install BuildBot and Twisted, createthe slaves, create directories and symlinks, download and copy tg_sbuildsteps.py and start slaves.</p>
<ol class="arabic simple">
<li>easy_install-2.4 virtualenv</li>
<li>mv /usr/bin/virtualenv /usr/bin/virtualenv-2.4</li>
<li>wget <a class="reference" href="http://peak.telecommunity.com/dist/ez_setup.py">http://peak.telecommunity.com/dist/ez_setup.py</a></li>
<li>/usr/local/bin/python2.5 ez_setup.py</li>
<li>/usr/local/bin/easy_install-2.5 virtualenv</li>
<li>mv /usr/local/bin/virtualenv /usr/local/bin/virtualenv-2.5</li>
<li>buildbot create-slave slave25 &lt;master&gt; &lt;name&gt; &lt;passwd&gt;</li>
<li>buildbot create-slave slave24 &lt;master&gt; &lt;name&gt; &lt;passwd&gt;</li>
</ol>
<p>The next three steps have to be done for both slave directories:
9. switch to slave directory
10. create symlink to virtualenv (ln -s /usr/(local/)bin/virtualenv-2.(4 or 5) virtualenv)
11. create directories</p>
<blockquote>
<ul class="simple">
<li>mkdir TG2full</li>
<li>mkdir TG2full/build</li>
<li>cd TG2full/build</li>
<li>mkdir pylons webhelpers</li>
</ul>
</blockquote>
<ol class="arabic simple" start="12">
<li>download tg_sbuildsteps.py from svn (<a class="reference" href="http://svn.turbogears.org/build/buildmaster/tg_sbuildsteps.py">http://svn.turbogears.org/build/buildmaster/tg_sbuildsteps.py</a>) and copy it to
(directory of your python installation)/lib/python25/site-packages/buildbot-0.7.7-py2.5.egg/buildbot/slaves.
Rename commands.py to _commands.py and run cat _commands.py tg_mbuildsteps.py &gt; commands.py</li>
</ol>
<p>So now you are finished. You have 2 slaves with you can start with buildbot start &lt;DirOfSlave&gt;.
If you're building dependencies there could be a problem. The first time you run a build I've got the error that the rsa key of some SVN repo is unknown and that you
have to add it manualy. In this case you have to  add it manual and start the build again. I will try to list the URLs you have
to add but if you get this error you know what to do.</p>
<div class="section">
<h3><a class="toc-backref" href="#id6" id="solaris-py23" name="solaris-py23">Solaris Py23</a></h3>
<p>I've stopped working on the Py23 slave for Solaris because I was not able to build Python2.3 on this OS. If I have time after the creation of the Windows slave, I create this slave.</p>
</div>
</div>
</div>
<div class="section">
<h1><a class="toc-backref" href="#id7" id="how-to-create-a-windows-slave" name="how-to-create-a-windows-slave">How-To create a windows slave</a></h1>
<p>To create a slave with Windows is easy. The how-to from buildbot.net works well. So let's start:</p>
<p>First download some installer:</p>
<ol class="arabic simple">
<li>Visit python.org and download the installer for Python 2.3, 2.4 and 2.5.</li>
<li>Download pyWin32 entensions from <a class="reference" href="https://sourceforge.net/projects/pywin32/">https://sourceforge.net/projects/pywin32/</a> and MinGW from MinGW <a class="reference" href="http://mingw.org">http://mingw.org</a></li>
<li>Download Twisted for Py2.4 from <a class="reference" href="http://twistedmatrix.com/">http://twistedmatrix.com/</a></li>
<li>Download BuildBot as a zipfile</li>
<li>Run all installer</li>
<li>extract buildbot zip and run the setup.py with the install argument with Py2.4.
Then you have to edit the buildbot.bat and change the python directroy to python24</li>
</ol>
<p>Edit the environment variables</p>
<ol class="arabic simple" start="7">
<li>add .py to PATHEXT</li>
<li>add the python24 and python24scripts directory to PATH</li>
</ol>
<p>Install easy_install and virtualenv</p>
<ol class="arabic simple" start="9">
<li>download eay_install from peak.telecommunity.com/dist/ez_setup.py</li>
<li>copy this script in all three python directories</li>
<li>open cmd and run this script with the different python versions</li>
<li>run easy_install virtualenv with the diffenrent python versions</li>
</ol>
<p>Create the slaves</p>
<ol class="arabic simple" start="13">
<li>buildbot create-slave slave25 &lt;master&gt; &lt;name&gt; &lt;passwd&gt;</li>
<li>buildbot create-slave slave24 &lt;master&gt; &lt;name&gt; &lt;passwd&gt;</li>
<li>buildbot create-slave slave23 &lt;master&gt; &lt;name&gt; &lt;passwd&gt;</li>
</ol>
<p>The creation of the slaves is similar to UNIX OS.</p>
<p>Then we have to create some files.
16. Like the symlinks on a unix-like OS, we need something comparable in Windows.</p>
<div class="system-message">
<p class="system-message-title">System Message: ERROR/3 (<tt class="docutils">buildslaves.rst</tt>, line 233)</p>
Unexpected indentation.</div>
<blockquote>
We create a batch file with this content and the name virtualenv.bat in every slave directory:
C:python2[3,4,5]scriptsvirtualenv %*</blockquote>
<div class="system-message">
<p class="system-message-title">System Message: WARNING/2 (<tt class="docutils">buildslaves.rst</tt>, line 235)</p>
Block quote ends without a blank line; unexpected unindent.</div>
<ol class="arabic simple" start="17">
<li>download tg_sbuildsteps.py from svn (<a class="reference" href="http://svn.turbogears.org/build/buildmaster/tg_sbuildsteps.py">http://svn.turbogears.org/build/buildmaster/tg_sbuildsteps.py</a>) and append its content
to the commands.py in lib/python25/site-packages/buildbot-0.7.7-py2.5.egg/buildbot/slaves of your Python24 directory.</li>
</ol>
<p>Our last step is to create the directories for the Mercurial checkouts.
We need a TG2fullbuildwebhelpers and a TG2fullbuildpylons directory in every slave directory.</p>
</div>
<div class="section">
<h1><a class="toc-backref" href="#id8" id="known-problems" name="known-problems">Known problems</a></h1>
<blockquote>
<ul class="simple">
<li>Buildbot needs a stable connection. It's not recommended to run a build slave with a WiFi connection. This will spam Buildbot's waterfall display with connect/disconnect messages.</li>
</ul>
</blockquote>
</div>
</div>
</body>
</html>
